{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Explore LAT Data\n",
    "\n",
    "It is recommended that the user first gain some basic familiarity with the structure and content of the [LAT data products](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_Data/LAT_DP.html), as well as the process for [preparing the data](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data_preparation.html) by making cuts on the data file. This tutorial demonstrates various ways of examining and manipulating the LAT data, but it is a simple approach useful for quickly exploring the data.\n",
    "\n",
    ">**IMPORTANT**! In almost all cases, light curves and energy spectra need to be produced by a [Likelihood Analysis](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/binned_likelihood_tutorial.html) using [gtlike](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtlike.txt) for robust results.\n",
    "\n",
    "In addition to the Fermitools, you will also be using the following FITS File Viewers:\n",
    "\n",
    "* _ds9_ (image viewer); download and install from: http://hea-www.harvard.edu/RD/ds9\n",
    "\n",
    "* _fv_ (view images and tables; can also make plots and histograms;\n",
    "download and install from: http://heasarc.gsfc.nasa.gov/docs/software/ftools/fv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Files\n",
    "\n",
    "Some of the files used in this tutorial were prepared within the [Data Preparation](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data_preparation.html) tutorial. These are:\n",
    "\n",
    "* `3C279_region_filtered_gti.fits` (16.6 MB) - a 20 degree region around the blazar 3C 279 with appropriate selection cuts applied\n",
    "* `spacecraft.fits` (67.6 MB) - the spacecraft data file for 3C 279.\n",
    "\n",
    "In the case that they have been lost, run these code cells below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget \"https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data/exploreLATData/3C279_region_filtered_gti.fits\"\n",
    "!wget \"https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data/exploreLATData/spacecraft.fits\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir ./data\n",
    "!mv *.fits ./data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls ./data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, you can select your own region and time period of interest from the [LAT data server](http://fermi.gsfc.nasa.gov/cgi-bin/ssc/LAT/LATDataQuery.cgi) and substitute them. **Photon** and **spacecraft** data files are all that you need for the analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Exploring the Data\n",
    "\n",
    "In this section we look at several different ways to explore the data and make quick plots and images of the data. First, we'll look at making quick counts maps with *ds9*; then we'll perform a more in depth exploration of the data using *fv*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### *ds9* Quick look:\n",
    "\n",
    "To see the data, use *ds9* to create a quick counts maps of the events in the file.\n",
    "\n",
    "For example, to look at the 3C 279 data file, type the following command in your command line:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "> ds9 -bin factor 0.1 0.1 -cmap b -scale sqrt 3C279_region_filtered_gti.fits &\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A *ds9* window will open up and an image similar to the one shown below will be displayed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Image,HTML\n",
    "HTML(\"<img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/ds9_quickview.png'>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Breaking the command line into its parts, we find that:\n",
    "\n",
    "* *ds9* - Invokes the command.\n",
    "\n",
    "\n",
    "* -bin factor 0.1 0.1 - Tells *ds9* that the x and y bin sizes are to be 0.1 units in each direction. Since we will be binning on the coordinates (RA, DEC), this means we will have 0.1 degree bins.\n",
    "\n",
    ">**Note**: The default factor is 1, so if you leave this off the command line you will get 1 degree bins\n",
    "\n",
    "* -cmap b - Tells ds9 to use the \"b\" color map to display the image. This is completely optional and the choice of map \"b\" represents the personal preference of the author. If left off, the default color map is \"gray\" (a grayscale color map).\n",
    "\n",
    "\n",
    "* -scale sqrt - Tells *ds9* to scale the colormap using the square root of the counts in the pixels. This particular scale helps to accentuate faint maxima where there is a bright source in the field as is the case here. Again this is the author's personal preference for this option. If left off, the default scale is linear.\n",
    "\n",
    "\n",
    "* & - backgrounds the task, allowing continued use of the command line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exploring with fv:\n",
    "\n",
    "*fv* gives you much more interactive control of how you explore the data.\n",
    "\n",
    "It can make plots and 1D and 2D histograms, allow you look at the data directly, and enable you to view the FITS file headers to look at some of the important keywords.\n",
    "\n",
    "Starting it up is easy; just type *fv* and the filename in the command line:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "> fv 3C279_region_filtered_gti.fits &\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(\"<img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/fv1.png'>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This will bring up two windows:\n",
    "- the general *fv* menu window (shown in cell above)\n",
    "- summary information about the file you are looking at (shown in cell below)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(\"<img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/fv2.png'>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the purposes of the tutorial, we will only be using the summary window, but feel free to explore the options in the main menu window as well.\n",
    "\n",
    "Looking at the summary window, notice that:\n",
    "\n",
    "1. There are three FITS extensions (primary, EVENTs & GTI). This is what should be there. If you don't see three, there is something wrong with your file.\n",
    "\n",
    "2. There are 168828 events in the filtered 3C279 file (the number of rows in the EVENTS extension), and 22 pieces of information (the number of columns) for each event.\n",
    "\n",
    "3. There are 2791 GTI entries.\n",
    "\n",
    "From this window, data can be viewed in different ways:\n",
    "\n",
    "* For each extension, the FITS header can be examined for keywords and their values.\n",
    "\n",
    "* Histograms and plots can be made of the data in the EVENTS and GTI extensions.\n",
    "\n",
    "* Data in the EVENTS or GTI extensions can also be viewed directly.\n",
    "\n",
    "Let's look at each of these in turn."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Viewing an Extension Header\n",
    "\n",
    "Click on the `Header` button for the EVENTS extension; a new window listing all the header keywords and their values for this extension will be displayed. Notice that the same information is presented that was shown in the summary window; namely that: the data is a binary table (XTENSION='BINTABLE'); there are 123857 entries (NAXIS2=123857); and there are 22 data values for each event (TFIELDS=22).\n",
    "\n",
    "In addition, there is information about the size (in bytes) of each row and the descriptions of each of the data fields contained in the table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(\"<img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/events_header1.png'>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you scroll down, you will find some other useful information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(\"<img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/events_header2.png'>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are some fairly important keywords on this screen, including:\n",
    "* **DATE** - The date the file was created.\n",
    "* **DATE-OBS** - The starting time, in UTC, of the data in the file.\n",
    "* **DATE-END** - The ending time, in UTC, of the data in the file.\n",
    "* **TSTART** - The equivalant of DATE-OBS but in Mission Elapsed Time (MET). Note: MET is the time system used in the event file to time tag all events.\n",
    "* **TSTOP** - The equivalant of DATE-END in MET.\n",
    "* **MJDREF** - The Modified Julian Date (MJD) of the zero point for MET. This corresponds to midnight, Jan. 1st, 2001 for FERMI\n",
    "\n",
    "Finally, as you continue scrolling to the bottom of the header, you will see:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(\"<img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/events_header3.png'>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This part of the header contains information about the data cuts that were made to extract the data. These are contained in the various DSS keywords. For a full description of the meaning of these values see the [DSS keyword page](https://fermi.gsfc.nasa.gov/ssc/dev/binned_analysis/dss_keywords.html). In this file the DSVAL1 keyword tells us which kind of event class has been used, the DSVAL2 keyword tells us that the data was extracted in a circular region, 20 degrees in radius, centered on RA=193.98 and DEC=-5.82. The DSVAL3 keyword shows us that the valid time range is defined by the GTIs. The DSVAL4 keywords shows the selected energy range in MeV, and DSVAL5 indicates that a zenith angle cut has been defined."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Making a Counts Map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*fv* can also be used to make a quick counts map to see what the region you extracted looks like. To do this:\n",
    "\n",
    "1. In the summary window for the FITS file, click on the **Hist** button for the EVENTS extension. A Histogram window will open.\n",
    ">**Note**: We use the histogram option rather than the plot option, as that would produce a scatter plot.\n",
    "\n",
    "2. From the X column's drop down menu in the column name field, select **RA**.\n",
    "fv will automatically fill in the TLMin, TLMax, Data Min and Data Max fields based on the header keywords and data values for that column. It will also make guesses for the values of the Min, Max and Bin Size fields.\n",
    "\n",
    "3. From the Y column's drop down menu in the column name field, select **DEC** from the list of columns.\n",
    "\n",
    "4. Select the limits on each of the coordinates in the Min and Max boxes.\n",
    "In this example, we've selected the limits to be just larger than the values in the Data Min and Data Max field for each column.\n",
    "\n",
    "5. Set the bin size for each column (in the units of the respective column; in this case we used 0.1 degrees)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(\"<img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/fv_quickview.png'>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this map, we've selected 0.1 degree bins."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6. You can also select a data column from the FITS file to use as a weight if you desire.\n",
    "For example, if you wanted to make an approximated flux map, you could select the ENERGY column in the Weight field and the counts would be weighted by their energy.\n",
    "\n",
    "\n",
    "7. Click on the \"Make\" button to generate the map.\n",
    "\n",
    "  This will create the plot in a new window and keep the histogram window open in case you want to make changes and create a different image. The \"Make/Close\" button will create the image and close the histogram window.\n",
    "\n",
    "  *fv* also allows you to adjust the color and scale, just as you can in *ds9*. However, it has a different selection of color maps.\n",
    "\n",
    "  As in *ds9*, the default is gray scale. The image at right was displayed with the cold color map, selected by clicking on the \"Colors\" menu item, then selecting: \"Continuous\" submenu --> \"cold\" check box."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Binning the Data\n",
    "\n",
    "While *ds9* and *fv* can be used to make quick look plots when exploring the data, they don't automatically do all the things you would like when making data files for analysis. For this, you will need to use the *Fermi*-specific [gtbin](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtbin.txt) tool to manipulate the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use [gtbin](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtbin.txt) to bin photon data into the following representations:\n",
    "\n",
    "* Images (maps)\n",
    "* Light curves\n",
    "* Energy spectra (PHA files)\n",
    "\n",
    "This has the advantage of creating the files in exactly the format needed by the other science tools as well as by other tools such as [XSPEC](http://heasarc.gsfc.nasa.gov/docs/xanadu/xspec/), and of adding correct WCS keywords to the images so that the coordinate systems are properly displayed when using image viewers (such as *ds9* and *fv*) that can correctly interpret the WCS keywords."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section we will use the [3C279_region_filtered_gti.fits](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data/exploreLATData/3C279_region_filtered_gti.fits) file to make images and will look at the results with *ds9*. In the [Explore LAT Data (for Burst)](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/explore_latdata_burst.html) section we will show how to use [gtbin](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtbin.txt) to produce a light curve.\n",
    "\n",
    "Just as with *fv* and *ds9*, *gtbin* can be used to make counts maps out of the extracted data.\n",
    "\n",
    "The main advantage of using [gtbin](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtbin.txt) is that it adds the proper header keywords so that the coordinate system is properly displayed as you move around the image.\n",
    "\n",
    "Here, we'll make the same image of the anti-center region that we make with *fv* and *ds9*, but this time we'll use the [gtbin](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtbin.txt) tool to make the image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[gtbin](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtbin.txt) is invoked on the command line with or without the name of the file you want to process. If no file name is given, [gtbin](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtbin.txt) will prompt for it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "gtbin\n",
    "    CMAP\n",
    "    ./data/3C279_region_filtered_gti.fits\n",
    "    ./data/3C279_region_cmap.fits\n",
    "    NONE\n",
    "    400\n",
    "    400\n",
    "    0.1\n",
    "    CEL\n",
    "    193.98\n",
    "    -5.82\n",
    "    0\n",
    "    AIT\n",
    "\n",
    "#### Parameters:\n",
    "# Type of output file (CCUBE|CMAP|LC|PHA1|PHA2)\n",
    "# Event data file name\n",
    "# Output file name\n",
    "# Spacecraft data file name [NONE is valid]\n",
    "# Size of the X axis in pixels\n",
    "# Size of the Y axis in pixels\n",
    "# Image scale (in degrees/pixel)\n",
    "# Coordinate system (CEL - celestial, GAL -galactic) (CEL|GAL)\n",
    "# First coordinate of image center in degrees (RA or galactic l)\n",
    "# Second coordinate of image center in degrees (DEC or galactic b)\n",
    "# Rotation angle of image axis, in degrees\n",
    "# Projection method e.g. AIT|ARC|CAR|GLS|MER|NCP|SIN|STG|TAN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are many different possible projection types. For a small region, the difference is small, but you should be aware of it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, we want a counts map, so we:\n",
    "1. Select `CMAP`.\n",
    "\n",
    "  The CCUBE (counts cube) option produces a set of count maps over several energy bins.\n",
    "  \n",
    "  \n",
    "2. Provide an **output file name**.\n",
    "3. Specify `NONE` for the spacecraft file as it is not needed for the counts map.\n",
    "4. Input **image size and scale** in pixels and degrees/pixel.\n",
    "\n",
    "  **Note**: We select a 400x400 pixel image with 0.1 degree pixels in order to create an image that contains all the extracted data.\n",
    "  \n",
    "  \n",
    "5. Enter the **coordinate system**, either celestial (CEL) or galactic (GAL), to be used in generating the image. The coordinates for the image center (next bullet) must be in the indicated coordinate system.\n",
    "6. Enter the **coordinates** for the center of the image, which here correspond to the position of 3C 279.\n",
    "7. Enter the **rotation angle (0)**.\n",
    "8. Enter the projection method for the image. See Calabretta & Greisen 2002, A&A, 395, 1077 for definitions of these projections.\n",
    "\n",
    "Here is the output [counts map file](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data/exploreLATData/3C279_region_cmap.fits) to use for comparison."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(\"<img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/gtbin_image.png'>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare this result to the images made with *fv* and *ds9* and you will notice that the image is flipped along the y-axis.\n",
    "\n",
    "This is because the coordinate system keywords have been properly added to the image header and the Right Ascension (RA) coordinate actual increases right to left and not left to right.\n",
    "\n",
    "Moving the cursor over the image now shows the RA and Dec of the cursor position in the FK5 fields in the top left section of the display.\n",
    "\n",
    "If you want to look at coordinates in another system, such as galactic coordinates, you can make the change by first selecting the '**WCS**' button (in the first row of buttons), and then the appropriate coordinate system from the choices that appear in the second row of buttons (FK4, FK5, IRCS, Galactic or Ecliptic)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Looking at the Exposure\n",
    "\n",
    "In this section, we explore ways of generating and looking at exposure maps. If you have not yet run [gtmktime](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtmktime.txt) on the data file you are examining, this analysis will likely yield incorrect results. It is advisable to prepare your data file properly by following the [Data Preparation](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data_preparation.html) tutorial before looking in detail at the [livetime and exposure](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_Data_Exploration/livetime_and_exposure.html).\n",
    "\n",
    "Generally, to look at the exposure you must:\n",
    "\n",
    "1. Make an livetime cube from the spacecraft data file using **gtltcube**.\n",
    "2. As necessary, merge multiple livetime cubes covering different time ranges.\n",
    "3. Create the exposure map using the gtexpmap tool.\n",
    "4. Examine the map using *ds9*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate the Livetime\n",
    "\n",
    "In order to determine the exposure for your source, you need to understand how much time the LAT has observed any given position on the sky at any given inclination angle.\n",
    "\n",
    "[gtltcube](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtltcube.txt) calculates this 'livetime cube' for the entire sky for the time range covered by the spacecraft file.\n",
    "\n",
    "To do this, you will need to make the livetime cube from the spacecraft (pointing and livetime history) file, using the [gtltcube](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtltcube.txt) tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "gtltcube\n",
    "    ./data/3C279_region_filtered_gti.fits\n",
    "    ./data/spacecraft.fits\n",
    "    ./data/3C279_region_ltcube.fits\n",
    "    0.025\n",
    "    1\n",
    "\n",
    "#### gtltcube Parameters:\n",
    "# Event data file\n",
    "# Spacecraft data file\n",
    "# Output file\n",
    "# Step size in cos(theta) (0.:1.)\n",
    "# Pixel size (degrees)\n",
    "#\n",
    "# May take a while to finish"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**gtltcube** may take some time to finish."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see below, the details recorded in the livetime cube file are multi-dimensional and difficult to visualize. For that, we will need an exposure map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(\"<img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/livetime_summary.png'>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combining _multiple_ livetime cubes\n",
    "\n",
    "In some cases, you will have multiple livetime cubes covering different periods of time that you wish to combine in order to examine the exposure over the entire time range.\n",
    "\n",
    "One example would be the researcher who generates weekly flux datapoints for light curves, and has the need to analyze the source significance over a larger time period. In this case, it is much less CPU-intensive to combine previously generated livetime cubes before calculating the exposure map, than to start the livetime cube generation from scratch.\n",
    "\n",
    "To combine multiple livetime cubes into a single cube use the [gtltsum](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtltsum.txt) tool."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**: [gtltsum](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtltsum.txt) is quick, but it does have a few limitations, including:\n",
    "* It will only add two cubes at a time. If you have more than one cube to add, you must do them one at a time.\n",
    "\n",
    "* It does not allow you append to or overwrite an existing livetime cube file.\n",
    "\n",
    "   For example: if you wanted to add four cubes (c1, c2, c3 and c4), you cannot add c1 and c2 to get cube_a, then add cube_a and c3 and save the result as cube_a; you must give it a different name.\n",
    "\n",
    "\n",
    "* The calculation parameters that were used to generate the livetime cubes (step size and pixel size) must be identical between the livetime cubes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is an example of adding two livetime cubes from the first and second halves of the six months of 3C 279 data using **gtltsum** (where the midpoint was 247477908 MET):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget \"https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data/exploreLATData/3C279_region_first_ltcube.fits\"\n",
    "!wget \"https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data/exploreLATData/3C279_region_second_ltcube.fits\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mv *cube.fits ./data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls ./data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "gtltsum\n",
    "    ./data/3C279_region_first_ltcube.fits\n",
    "    ./data/3C279_region_second_ltcube.fits\n",
    "    ./data/3C279_region_summed_ltcube.fits\n",
    "\n",
    "#### Parameters:\n",
    "# Livetime cube 1 or list of files\n",
    "# Livetime cube 2\n",
    "# Output file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate an Exposure Map or Cube\n",
    "\n",
    "Once you have a livetime cube for the entire dataset, you need to calculate the exposure for your dataset. This can be in the form of an exposure **map** or an exposure **cube**.\n",
    "\n",
    "* Exposure **maps** are mono-energetic, and each plane represents the exposure at the midpoint of the energy band, not integrated over the band's energy range. Exposure maps are used for **unbinned** analysis methods. You will specify the number of energy bands when you run the [gtexpmap](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtexpmap.txt) tool.\n",
    "\n",
    "* Exposure **cubes** are used for **binned** analysis methods. The binning in both position and energy must match the binning of the input data file, which will be a counts cube. When you run the [gtexpcube2](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtexpcube2.txt) tool, you must be sure the binning matches.\n",
    "\n",
    "For simplicity, we will generate an exposure map by running the [gtexpmap](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtexpmap.txt) tool on the event file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[**gtexpmap**](https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/help/gtexpmap.txt) allows you to control the exposure map parameters, including:\n",
    "\n",
    "* Map center, size, and scale\n",
    "* Projection type (selection includes Aitoff, Cartesian, Mercator, Tangential, etc.; default is Aitoff)\n",
    "* Energy range\n",
    "* Number of energy bins\n",
    "\n",
    "The following example shows input and output for generating an exposure map for the region surrounding 3C 279."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "gtexpmap\n",
    "    ./data/3C279_region_filtered_gti.fits\n",
    "    ./data/spacecraft.fits\n",
    "    ./data/3C279_region_summed_ltcube.fits\n",
    "    ./data/3C279_exposure_map.fits\n",
    "    P8R3_SOURCE_V2\n",
    "    30\n",
    "    500\n",
    "    500\n",
    "    30\n",
    "\n",
    "#### gtexpmap Parameters: ALSO SEE BELOW\n",
    "# Event data file\n",
    "# Spacecraft data file\n",
    "# Exposure hypercube file\n",
    "# Output file name\n",
    "# Response functions\n",
    "# Radius of the source region (in degrees)\n",
    "# Number of longitude points (2:1000)\n",
    "# Number of latitude points (2:1000)\n",
    "# Number of energies (2:100)\n",
    "\n",
    "# This will generate an exposure map on six months of data.\n",
    "# This may take a long time.\n",
    "# Below you will find a wget command to get the resulting file."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**As six months is far too much data for an unbinned analysis, this computation will take a long time. Skip past this section to find a copy of the output file.**\n",
    "\n",
    "Running **gtexpmap** in the command line will prompt you to enter the following:\n",
    "\n",
    "* Name of an events file to determine the energy range to use.\n",
    "* Name of the exposure cube file to use.\n",
    "* Name of the output file and the instrument response function to use.\n",
    "    * For more discussion on the proper instrument response function (IRF) to use in your data analysis, see the overview discussion in the [Cicerone](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_LAT_IRFs/IRF_overview.html), as well as the current [recommended data selection](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_Data_Exploration/Data_preparation.html) information from the LAT team. The [LAT data caveats](https://fermi.gsfc.nasa.gov/ssc/data/analysis/LAT_caveats.html) are also important to review before starting LAT analysis.\n",
    "\n",
    "The next set of parameters specify the size, scale, and position of the map to generate.\n",
    "\n",
    "* The radius of the 'source region'\n",
    "    * The source region is different than the region of interest (ROI). This is the region that you will model when fitting your data. As every region of the sky that contains sources will also have adjacent regions containing sources, it is advisable to model an area larger than that covered by your dataset. Here we have increased the source region by an additional 10°, which is the minimum needed for an actual analysis. Be aware of what sources may be near your region, and model them if appropriate (especially if they are very bright in gamma rays).\n",
    "* Number of longitude and latitude points\n",
    "* Number of energy bins that will have maps created\n",
    "    * This number can be small (∼5) for sources with flat spectra in the LAT regime. However, for sources like pulsars that vary in flux significantly over the LAT energy range, a larger number of energies is recommended, typically 10 per decade in energy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the output exposure map\n",
    "!wget \"https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/data/exploreLATData/3C279_exposure_map.fits\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mv 3C279_exposure_map.fits ./data/3C279_exposure_map.fits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!ls ./data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the file has been generated, it can be viewed with *ds9*. When you open the file in ds9, a \"Data Cube\" window will appear, allowing you to select between the various maps generated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML(\"<img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/ds9_datacube.png'>\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below are four of the 30 layers in the map, scaled by `log(Energy)`, and extracted from *ds9*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"First Layer on the left. Fourth layer on the right.\")\n",
    "display(HTML(\"<table><tr><td><img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/Expmap1.png'></td><td><img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/Expmap4.png'></td></tr></table>\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Tenth layer on the left. Last layer on the right.\")\n",
    "display(HTML(\"<table><tr><td><img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/Expmap10.png'></td><td><img src='https://fermi.gsfc.nasa.gov/ssc/data/analysis/scitools/images/explore_data/Expmap30.png'></td></tr></table>\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the exposure changes as you go to higher energies. This is due to two effects:\n",
    "\n",
    "1. The PSF increases at lower energies, making the wings of the exposure expand well outside the region of interest. This is why it is necessary to add at least 10 degrees to your ROI. In this case, you can see that even 10 degrees has not captured the full [effect of the PSF](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_LAT_IRFs/IRF_PSF.html).\n",
    "\n",
    "\n",
    "2. The change in the [effective area](https://fermi.gsfc.nasa.gov/ssc/data/analysis/documentation/Cicerone/Cicerone_LAT_IRFs/IRF_EA.html) of the LAT instrument as you go to higher energies.\n",
    "\n",
    "Both of these effects are quantified on the [LAT performance page](http://www.slac.stanford.edu/exp/glast/groups/canda/lat_Performance.htm)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
